Skip to content

[PECOBLR-421] Add explicit null check for empty values in an Arrow value vector#827

Merged
gopalldb merged 34 commits into
mainfrom
gopalldb/null-arrow
May 15, 2025
Merged

[PECOBLR-421] Add explicit null check for empty values in an Arrow value vector#827
gopalldb merged 34 commits into
mainfrom
gopalldb/null-arrow

Conversation

@gopalldb

Copy link
Copy Markdown
Collaborator

Description

Added explicit null check for empty value in an Arrow value vector. This issue comes when Arrow null check is disabled for optimized performance.

Testing

Added unit tests

Additional Notes to the Reviewer

// The zero value should still be correctly identified as 0, not null
assertFalse(vector.isNull(2));
assertEquals((byte) 0, convert(vector, 2, ColumnInfoTypeName.BYTE, "BYTE"));
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests are not actually testing any change in behaviour in our class, can we instead do verify calls to ensure the appropriate methods are being called when the value vector is null?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not simulating iceberg library usage. Need to find a better test case for that. Will handle in separate PR.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test now.

@samikshya-db

Copy link
Copy Markdown
Collaborator

In the description, can you add the stack trace faced by the customer? And how did we verify that this fix is working ?

@gopalldb

Copy link
Copy Markdown
Collaborator Author

In the description, can you add the stack trace faced by the customer? And how did we verify that this fix is working ?

Verified by adding test.

@gopalldb gopalldb merged commit 5e4934e into databricks:main May 15, 2025
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants